Use CRT response file in low level CRT S3AsyncClient for getObject #6289
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alextwoods
commented
Jul 22, 2025
...ests/src/test/java/software/amazon/awssdk/archtests/CodingConventionWithSuppressionTest.java
Show resolved
Hide resolved
alextwoods
changed the title
There was a problem hiding this comment.
Pull Request Overview
This PR implements support for using CRT's response file feature in the low-level CRT S3AsyncClient for getObject operations. This enhancement allows the CRT library to write response data directly to a file, improving memory usage by avoiding passing data chunks through Java.
Key changes:
- Added a new
getObject(GetObjectRequest, Path)method to DefaultS3CrtAsyncClient that uses CRT's direct file writing - Implemented execution attribute plumbing to pass response file path and options through the request pipeline
- Created a new response transformer that acts as a no-op when CRT handles file writing directly
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| DefaultS3CrtAsyncClient.java | Added getObject method with Path parameter and execution attribute mapping |
| S3CrtAsyncHttpClient.java | Enhanced to handle response file path and options from execution attributes |
| CrtResponseFileResponseTransformer.java | New no-op response transformer for CRT-managed file responses |
| S3InternalSdkHttpExecutionAttribute.java | Added new execution attributes for response file path and options |
| S3CrtAsyncHttpClientTest.java | Added test coverage for response file functionality |
| CrtResponseFileResponseTransformerTest.java | Comprehensive test suite for the new response transformer |
| CodingConventionWithSuppressionTest.java | Updated suppressions to include new transformer class |
| feature-AWSS3-233f74c.json | Changelog entry for the new feature |
.../src/test/java/software/amazon/awssdk/services/s3/internal/crt/S3CrtAsyncHttpClientTest.java
Outdated
Show resolved
Hide resolved
.../src/test/java/software/amazon/awssdk/services/s3/internal/crt/S3CrtAsyncHttpClientTest.java
Outdated
Show resolved
Hide resolved
...java/software/amazon/awssdk/services/s3/internal/crt/CrtResponseFileResponseTransformer.java
Show resolved
Hide resolved
zoewangg
reviewed
Jul 23, 2025
...3/src/main/java/software/amazon/awssdk/services/s3/internal/crt/DefaultS3CrtAsyncClient.java
Show resolved
Hide resolved
...java/software/amazon/awssdk/services/s3/internal/crt/CrtResponseFileResponseTransformer.java
Show resolved
Hide resolved
|
zoewangg
reviewed
Jul 28, 2025
...java/software/amazon/awssdk/services/s3/internal/crt/CrtResponseFileResponseTransformer.java
Show resolved
Hide resolved
zoewangg
approved these changes
Jul 28, 2025
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
|
This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use CRT response file in low level CRT S3AsyncClient for getObject.
Motivation and Context
awslabs/aws-crt-java#825 added support for setting the response file and allowing CRT to write directly to the file. That can improve memory usage by avoiding passing the chunks into java where they are then written to the file.
Note that this change does not change the Transfrer manager behavior - TransferManager will still use the Java based in memory transform.
Modifications
This follows the implementation for requestPath (the inverse feature, allowing paths to be set for putObject, see: #4379).
Testing
New and existing s3 tests, including TM + regression tests.
Screenshots (if appropriate)
Types of changes
Checklist
mvn installsucceedsscripts/new-changescript and following the instructions. Commit the new file created by the script in.changes/next-releasewith your changes.License